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ABSTRACT 

The application of the Minnesota Adaptive 
Instructional System (MAIS) decision procedure by R. D. Tennyson et 
al. (1975, 1977) is examined. The MAIS is a computer-based adaptive 
instructional system. The problems of determining the optimal number 
of interrogatory examples in the MAIS can be formalized as a problem 
of Bayesian decision making. Two features of the MAIS decision 
procedure can be improved by using other results from this 
decision-theory approach. The first feature deals with the 
determination of the loss ratio "R. ,f A lottery method for assessing 
this ratio empirically is discussed. The second feature concerns the 
choice of the loss function involved, it is argued that in many 
situations, the assumed threshold loss function in the MAIS is an 
unrealistic representation of the loss actually incurred. A linear 
utility function is proposed to meet the objections to threshold 
loss. Whether or not these two innovations are really improvements of 
the present decision component in the MAIS in terms of student 
performance on posttests, learning time, and amount of instruction 
must be decided on the basis of experiments. Research projects for 
these areas have already been planned. One table and one figure 
illustrate the decision theory approach. A 38-item list of references 
is included. (SLD) 
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Abstract 

The purpose of this paper is to review the Minnesota Adaptive 
Instructional System (MAIS) . It will first be indicated how 
the problem of determining the optimal number of 
interrogatory examples in MAIS can be formalized as a problem 
of Bayesian decision making. Subsequently, it will be shown 
how two features of MAIS can be improved by using other 
results from this decision— theoretic approach. The first 
feature deals with the determination of the loss ratio R. A 
lottery method for assessing empirically this ratio is 
discussed. The second feature concerns the choice of the loss 
function involved. It is argued that in many situations the 
assumed threshold loss function in MAIS is an unrealistic 
representation of the loss actually incurred. In view of 
this, a linear utility function is proposed. Finally, some 
new lines of research are suggested arising from the 
application of decision theory to the MAIS model. 
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The Use of Decision Theory in the 
Minnesota Adaptive Instructional System 

Introduction 

The term "adaptive instruction" has been in widespread use 
for over a decade (Farley, 1983,; Hansen, Ross & Rakow, 1977; 
Holland, 1977; Landa, 1976; Park, 1982; Tennyson & Breuer, 
1984). Although different authors have defined the term in a 
different way, most agree that it denotes the use of 
strategies to adapt instructional treatments to the changing 
nature of student abilities and characteristics during the 
learning process (Tennyson & Park, 1984). In the context of 
computer-based instruction (CBI),, adaptive instructional 
programs are often qualified as individualized study systems 
(ISS). Examples are the Pittsburgh Individually Prescribed 
Instruction (IPI) project (Glaser, 1968) and Computer- 
Assisted Instruction (CAI) (Atkinson, 1968- Suppes, 1966). 

In a special JCBI issue on Educational Research and 
Computer-Iias^d Instruction, Tennyson, Christensen and Park 
(1984) describe a computer-based adaptive instructional 
system, the Minnesota Adaptive Instructional System (MAIS). 
The authors call the system an intelligent CBI-system, 
because it exhibits some of machine intelligence, as 
demonstrated by its ability to improve decision making over 
the history of the system as a function of accumulated 
information about previous students. In the literature many 
successful research projects have been reported on MAIS 
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(e.g. , Park & Tennyson, 1980; Tennyson, Tennyson & Rothen, 
1960). 

Initial work on MAIS began as an attempt to design an 
adaptive instructional strategy for concept-learning 
(Tennyson, 1975). According to Merrill and Tennyson (1977) 
concept— learning can be conceived as a two-stage process of 
formation of conceptual knowledge and development of 
procedural knowledge (see Tennyson and Cocchi arella, 1986, 
for a complete review of the theory of concept-learning) . 

In MAIS . eight basic instructional design variables 
directly related to specific learning processes are 
distinguished. In order to adapt instruction to individual 
learner differences (aptitudes, prior knowledge) and learning 
needs (amount and sequence of instruction), these variables 
are controlled by an intelligent tutor system (Tennyson and 
Christensen. 1986). Three of these variables are directly 
managed by a computer-based decision strategy. The functional 
operation of this strategy was related to guidelines 
described by No^ick and Lewis (1974). 

The purpose of this paper is to review the application 
of the MAIS decision procedure by Tennyson and his 
associates. First, it will be indicated how this procedure 
can be situated within the general framework of (empirical ) 
Bayesian decision theory (e.g., DeGroot , 1970;, Ferguson, 
1967; Keeney & Raiffa,, 1976; Lindgren, 1976), and, what 
implicit assumptions have to be made in doing so. 

Using a Bayesian approach, the decision component in 
MAIS can be inmproved. As an example, it will be shown how 
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two features of MAIS can be improved by using other results 
from decision theory. The first feature is the determination 
of the loss ratio R. The second feature is the use of the 
linear utility function instead of the assumed threshold loss 
function in MAIS. 

The paper concludes with some new lines of research 
arising from the application of decision theory to the MAIS 
model. We shall confine ourselves in this paper only to one 
of the three instructional design variables directly managed 
by the decision component in MAIS, namely determining the 
optimal number of interrogatory examples (question form). 

A Framework of Bayesian Decision Theory 

The derivation of an optimal strategy with respect to the 
number of interrogatory examples in a concept-learning lesson 
requires an instructional problem be stated in a form 
amenable to a decision-theoretic analysis Analyses bas^d on 
decision theory vary somewhat from field to field, but the 
following formal elements can be found in most of them: 

1. A nonempty set, 0,, of possible states of nature, 

2. A nonempty set, ft, of actions available to the decision- 
maker . 

3. A loss function, l(a,8), i.e., a real-valued function 
defined on ft x 0. 
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4. A probability function or psychometric model, f(::|0), 
relating observed values x of a stochastic variable X to 
a given value 0=0 for the state of nature. 



These basic elements have been related to decision 
problems in educational testing by many authors, particularly 
in the context of computer-based adaptive instructional 
systems (e.g. , Atkinson , 1976 ; Swaminathan , Hambleton & 
Algina, 1975; van der Linden, 1981). As the use of the 
decision component in MAIS refers to sequential mastery 
testing, we shall discuss here only the application o f the 
basic elements to this problem. 

The first element concerns the student's true level of 
functioning it e [0,1]. In the present problem, there are two 
possible states of nature: a student is a true master (0^) if 
his/her true level of functioning exceeds the criterion level 
T^O € [0,1], and he/she is a true nonmaster (0q) otherwise. 
The criterion level irg-the minimum degree of mastery 
required-is- set 1:1 advance Unfortunately, due to measurement 
and sampling errors, the true level of functioning is 
unknown. All that is known is the student's observed test 
score X from a small sample of n interrogatory examples (x = 
0,1 n). 

The second element pertains to the following two 
available actions: advance a student (a^) to the next concept 
if his/her test score x exceeds a certain cut-off score c on 
the observed test score scale X, and retain (aQ) him/her 
otherwise. Students with test score x below the cut-off score 
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c are provided with additional expository examples (statement 
form). A new interrogatory example is then generated. This 
procedure is applied sequentially until either mastery is 
attained or the pool of test items is exhausted. Now. the 
sequential mastery decision problem can be stated as choosing 
a value of c that, given the value of ttq, is optimal in some 
sense. 

The third element describes the loss l(a^ t 0j) incurred 
when action a± (i=0,l) is taken for the student who is in 
state 9j (j=0,l). The loss must be measured on at least an 
interval scale. In Tennyson's approach the loss function is 
supposed to be a threshold function. The implicit choice of 
this function implies that the "seriousness" of all possible 
consequences of the two available actions can be summarized 
by four constants, one for each of the four possible decision 
outcomes (see Table 1). 



Insert Table 1 about here 



In Table 1 it is assumed that no losses occur for 
correct decisions and, therefore, the losses associated with 
coirect advance and retain decisions On and loo* 
respectively) are set equal to zero. 

In the decision component of MAIS, a loss ratio R must 
be specified. R refers to the relative losses associated with 
advancing a learner whose true level of functioning is below 

ERIC 11 



The Use of Decision Theory 

8 

ttq and retaining one whose true level exceeds ttq. From Table 
1 it can be seen that the loss ratio R equals lio/loi ^ or a H 
values of it. 

The las element relates the observed test score X to 
the true level of functioning tr . In MAIS this is done by 
using the beta-binomial model. 

f(x|TT) = (£) TT X (l-TT) n - X . 

Within a Bayesian decision-theoretic framework the 
sequential mastery dec j s ion problem is solved by minimizing 
the "Bayes risk" , which is minimal if for each value x ot X 
an action with smallest posterior expected loss is chosen. 
The posterior expected loss is the expected loss taken with 
respect to the posterior distribution of tr. 

It can be seen from the loss table that the decision 
rule which minimizes posterior expected loss is to advance a 
student whose test score x is such that 

loiProbdr > irojx.n) ^ lio Prob ^ 11 < TTol x - n) « 

and to retain him/her otherwise. Since Iqi > 0, this is 
equivalent to advancing a student if 

Prob(ir > Tr 0 |x,n) > R/(l+R). 

and retaining him/her otherwise. Prob(tr > tro|x.n) denotes the 
probability of the student's true level of funct'oning 



12 



The Use of Decision Theory 

9 



exceeding ttq, given his/her test score x on a test of length 
n. In fact, this probability is one minus the cumulative 
posterior distribution of ir. In MAIS this quantity is called 
the "beta value" or "operating level" v Tennyson, Christansen 
& Park, 1984) . 

It should be noted that, as can be seen from the 
decision rule, the decision-maker need not specify I^q and 
IqI in any absolute value. He need only specify their ratio 

ho/ioi- 

In order to initiate the decision component in MAIS, 
three kinds of parameters must be specified in advance 
(Rothen & Tennyson, 1984). Beside the parameters ttq and k, a 
probability distributj ^n representing the prior knowledge 
about tt must be available. In MAIS a beta distribution, 
B(cx,P), is used as a prior distribution and a pretest score 
together with information about other students is used to 
specify its parameter values. It follows that the posterior 
distribution of v is easily obtained. From an application of 
Bayes's theorem, the posterior distribution will again be a 
member of the beta family (the conjugacy property). In fact, 
if the prior distribution is B(a,P) and the student's test 
score is x from a test of length n, then the posterior 
distribution is B(x+a,n-x+p ) . The beta distribution is 
extensively tabulated (e.g. , Pearson, 1930). Normal 
approximations are also available (Johnson & Kotz, 1970, 
sect. 2.4.6). 

The MAIS decision procedure for adapting the number of 
interrogatory examples can now be summarized as follows: If a 
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student's beta value exceeds the quantity R/(l+R), (s)he is 
passed to the next instructional unit (i.e., next concept) or 
final (summative) posttest . However, if his/her beta value is 
below this quantity, his/her posterior distribution is used 
as a prior distribution in a next cycle. A new interrogatory 
example is then generated. The procedure is applied 
iteratively until either the beta value exceeds the quantity 
R/(l+R) or all interrogatory examples in the pool for that 
particular concept have been presented. 

In the next two sections two features of the MAIS 
decision procedure will be critically reviewed. Subsequently, 
it will be shown how they can be improved by using other 
results from decision theory. 



Determination of the Loss Ratio 



We start witn a point already raised by Tennyson and Park 
(1979) themselves. It concerns the manner in which the loss 
ratio is determined. In fact. MAIS does not include any 
techniques to determine the loss ratio. It only assumes that, 
prior to administering the pretest, the loss ratio is 
adjusted on the basis of a task-related aptitude score. A 
high score on the aptitude score is used to adjust the ratio 
so that the losses associated with the error of a false 
retain are decreased relative to the error of a false 
advancement (Rothen & Tennyson. 1978). However, as Tennyson 
and Park (1979) have pointed out. this procedure seems to be 

ERIC 1 4 
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without any theoretical base. In particular, a loss function 
should be independent upon the test data, 

As mentioned before, a loss function specifies the total 
costs of all possible decision outcomes . These costs Tiay 
concern all relevant psychological, social, and economic 
consequences which the decision brings along. An example of 
economic consequences is extra computer time associated with 
presenting additional instructional materials. 

For assessing loss functions, or the more generally 
applicable utility functions, most texts on decision theory 
propose lottery methods (see, for example, Luce & Raiffa, 
1957, Chap. 2; for a recent modification, see Novick & 
Lindley, 1979). But in principle any psychological scaling 
method can be used. Although helpful techniques are 
available, this does not mean that, for example, in programs 
of individualized instruction, assessment of utilities is 
always a simple matter. In this paper, we shall consider one 
method that works in decision problems with a finite number 
of outcomes such as in the sequential mastery decision 
problem. 

Generally speaking, utility theory uses the notions of 
desirability of outcomes to scale the consequences of each 
pair of action and state of nature. Surely, the most 
desirable outcomes in MAIS are the true positives and true 
negatives. In either case, we correctly classify the student. 
For this reason, it is assumed that both outcomes are equally 
preferred. Furthermore, if a true nonmaster is advanced, 
(s)he will not only lose the time required to complete the 
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next concept, but may also become frustraced and discouraged. 
Therefore, let us assume that misclassif ying a true master is 
much more desirable than misclassifyinc, a true nonmaster . 

The set of the possible outcomes associated with making 
decision a^ when 8j is the true state of nature, will be 
represented by O^j. Let 0 max be the most preferred outcome 
and 0 m i n be the least preferred outcome in this set. So, in 
our notation, we have Oqo = °11 = tynax and Oiq = 0 miu . Let 
u ij describe the utility of 0^ j . These utilities are measured 
numerically on an interval scale by the following device. 

Leu u max and u min , the utilities of the most preferred 
and least preferred outcome of the possible outcomes, be 
assigned values 1 and 0. respectively. Further, suppose the 
decision— maker is indifferent between Oqi for certain and a 
conceptual lottery which has probability p of realizing 0 max 
and Cl-p) of realizing 0 min . It is assumed (Luce & Raiffa, 
1957, Chap. 2) that this indifference will occur if and only 
if the person has the same utility for Oqi for sure and for 
the specified lottery. The expected utility of the lottery is 
then p.u max + (l-p)u miri which equals p. since u max = 1, 
u min = 0. Hence the utility of Oqi is operationally defined 
as p. that is uqi = p. This procedure can then be followed 
for each O^j in turn until utilities have been assigned to 
each of the possible outcomes. 

As an aside, we note that for each possible outcome 0^ 
one can always define a suitable loss by taking the 
difference between the utility of the most preferred outcome 
and Oji CLindley, 1972) . 
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Admittedly, specifying utility functions is not an easy 
task, but practical experience with the above method shows it 
can be done (e.g., Vrijhof, Mellenbergh & van den Brink, 
1983). If, for example, correctly classifying a student gives 
our decisionmaker 10 "utiles" more than misclassifying a 
true nonmaster and only 5 "utiles" more than misclassifying a 
true master, then uqi would be 0.5. This means that 
misclassifying a true master is half-way between 
misclassifying a true nonmaster and correctly classifying a 
student on an interval utility scale. It follows that 

R = lio/loi = (u max - u 10>/( u max - u 01> = 2 - 

In the MAIS decision procedure , it is assumed that the 
form of the utility structure involved is a threshold 
function. Therefore, only uqi has to be assessed empirically. 
Us ing the described techniques , however , a dec is ionnnaker ' s 
utility structure can be completely assessed without making 
any assumptions about the form of the utility function. Only 
minimal axioms from utility theory have to be assumed. 
However, as van der Linden (1981) has pointed out, these 
techniques do not automatically lead to elegant utility 
functions and optimal cutting scores. It may be wise, 
therefore, to i>se these techniques only for a prior chosen 
mathematical form of the utility function. In addition to 
threshold loss function, however, other more useful functions 
have been adopted in decision theory. One such function will 
be considered below. 
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A Linear Utility Function 



An obvious disadvantage of the threshold utility function is 
that it assumes constant utility for students to the left or 
to the right of ttq, no matter how large their distance from 
ttq is. For instance, a misclassif ied true master with a true 
level of functioning just above ttq gives the same utility as 
a misclassif ied true master with a true level far above ttq. 
It seems more realistic to suppose that for misclassif ied 
true masters the utility is a monotonically decreasing 
function of the variable tr. 

Moreover, as can be seen in Table 1, the threshold 
utility function shows a "threshold" at the point tr = ttq, and 
this also seems unrealistic in many cases . In the 
neighbourhood of this point, the utilities for correct and 
incorrect decisions frequently change smoothly rather than 
abruptly. 

In view of this, van der Linden and Mellenbergh (1977) 
propose a linear utility function: 



The above defined function consists of a constant term 
and a term proportional to the difference between the true 



bodrO-tO+dO 



for retain (aQ) 



u(ai.tr) = < 



bo, bi > 0 



bi(tr-tro)+di 



for advance (ai ) 
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level of functioning *rr and the specified criterion level irg. 
The c ons t ant amount s of utility, dj ( j =0 , 1) , can, for 
example, represent the utility of testing, which will be 
mostly negative because costs of testing are involved. The 
condition bg. b^ > 0 is equivalent to the statement that for 
actions ag and a^, utility is a strictly decreasing and 
increasing function of the variable tr, respectively. The 
parameters bg, b^, dg, and d^ have to be assessed 
empirically. Figure 1 displays an example of this function. 



Insert Figure 1 about here 



As the general linear utility function now stands, we 
need to determine the four constants bg, bj_. dg , and d^ 
before it can be applied. However, if we use the fact that a 
utility function needs to determined only up to a positive 
multiplicative constant and an additive constant (e.g., Luce 
& Raiffa. 1957). we can reduce the number of unknown 
constants to two. Thus, since b^ > 0, we may redefine u(a^.iT) 
by making the positive linear transformation u^Ca^.Tr) = 
[u( a it ir )-di] /b]_. And so 



u*(ai,tr) = 



b*(iro-ir)+d* ,1=0 



TT - TTO , i = 1 
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where b* = bg/bi and d* = (dQ-d^/bj . 

We turn now to an illustration of one of the most direct 
methods available for determining the constants b* and d* . In 
order to make the method work, the decision-maker must be 
able to specify two ordered pairs (^i.^) and such 
that 

u*(ao.ir i ) = u*(ai.TTj) 

and 

u*(a0,frj) = u*(aj .tfj) . 
Solving this system of equations, we find that 

b = ("irj-TT.j)/(TT i -'tri) and d = tTj-tro"" 15 (*0 -ir i)- 

Analogous to the minimization of posterior expected 
loss, decision theory with a utility function requires us to 
select that action which will maximize the posterior expected 
utility So., the decision rule that maximizes posterior 
expected utility in the case of a linear utility function is 
to advance a student with test score x for which 

E [ ( tt — ttq ) | x , n] > Et(b*(ir 0 -Tr)+d*|x.n] 

and to retain him/her otherwise. Since (1+b*) > 0. this is 
equivalent to advancing a student if 

ERIC 20 
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E[ir|x.n] £ Tr 0 +d*/(l+b*) . 

and retaining him/her otherwise. In other words, with linear 
utility, the action taken depends only upon the expectation 
of the posterior distribution of tr, other attributes of the 
distribution are irrelevant for decision purposes. 

Using the fact that the expectation of a beta 
distribution B(oc,p) is equal to a/(a+p), and. thus, the 
posterior expectation equals (oc+x)/(a+x+p+n-x) , it follows 
that a student is advanced if his/her test score x is such 
that 

x > [oc+p+n] [ir 0 +d*/O+b*)]-a, 

and retained otherwise. 

Putting u*(a0.tr) and u*(a lf ir) equal to each other., it 
appears that the it-coordinate of the intersection of both 
utility lines is equal to Trg+d*/(l+b* ) . Therefore, the 
decision rule can be viewed as advancing a student if his/her 
expectation of the posterior distribution of it is to the 
right of the intersection point, and retaining him/her 
otherwj.se . 

When d* =0. that is dg = dj_ t both utility lines 
intersect at tr = ttq and an interesting case arises. Then, all 
utility function parameters vanish from the decision rule, 
and thus, takes the form of advancing a student if 

E[tr |x,n] > ttq . 
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and retaining him/her otherwise. In other words, if the 
amounts of constant utility, d j , for both decisions are equal 
or if there are no constant utilities at all . there is no 
need to assess the parameters d* and b* in adapting the 
number of interrogatory examples. In that case, the decision 
rule can even be simplified to advance a student if his/her 
expectation of the posterior distribution of tr is greater 
than or equal to the specified criterion level ttq. and to 
retain him/her otherwise. 

New Lines of Research 

There are a few new lines of research arising from the 
application of decision theory to the decision component in 
MAIS. The first is the extension of the work of Tennyson and 
his associates to situations where guessing and carelessness 
are incorporated. Morgan (1979) has developed a model with 
corrections for guessing antf carelessness within a Bayesian 
decision-theoretic framework. The results of a computer 
simulation of the model indicate that guessing and 
carelessness may markedly affect ti e determination of cutting 
scores, and hence the accuracy of decisions about mastery. 

The second line is research into other prior 
distributions about ir (for example, the standard normal 
distribution) than the beta prior assumed in MAIS. It might 
also be assumed that no prior distribution about tr is 
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available, because specifying such a distribution is too 
difficult a job to accomplish. In these circumstances, the 
minimax procedure may be an appropriate framework (e.g.. 
Huynh, 1980; van der Linden, 1981) which requires no prior 
distribution regarding the true level of functioning. In this 
case, the optimum cutting score is obtained by minimizing the 
maximum risk which would incurred by misclassif ications . 

Finally, an interesting new line of research seems to be 
an extension of the action space fi. In MAIS, two actions were 
available to the decision-maker, namely advancing (ajl or 
retaining (ag) a student. However, it might also be 
hypothesized that there are three (or any finite number) of 
actions open to the decision-maker. 

For example, in the three-action problem the student may 
provided with additional instructional materials both of the 
present and the previous concept (a2) ; (s)he may provided 
only with additional instructional materials of the present 
concept (aQ); oi , (s)he tray advance to the next concept (ai). 

We might think of this problem in terms of specifying 
two cutting test scores cq and c^ on the observed test score 
scale X, where eg < c^ . Then for observed test score x < eg, 
action a2 will be taken; for eg < x < Cj, action ag will be 
taken; and, for 

x > Cj, action aj will be taken. 

Davis, Hickman and Novick (1973) have given a solution 
to the three-action problem by using natural extensions of 
the threshold loss function. Although the notation becomes 
more complex and the computation a bit more tedious, there 
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are no fundamentally new ideas in the multiple action 
problem. 



In this paper it was indicated how the MAIS decision 
procedure can be formalized within a Bayesian decision- 
theoretic framework by epplying the basic elements of the 
theory to the instructional decision of determining the 
optimal number of interrogatory examples in MAIS. In fact, it 
turned out that this decision can be considered as a 
sequential mastery decision. 

Moreover, it was the purpose of this paper to show how 
two features of MAIS can be improved as a consequence of this 
formalization by using other results from decision theory. 
The first feature is the iete: mination of the loss ratio R. A 
lottery method has been discussed for assessing this ratio 
empirically. The second feature is the threshold loss 
function. It was argued that in many situations this is an 
unrealistic representation of the loss actually incurred. 
Instead, a linear utility function was proposed to meet the 
objections to threshold loss. 

Whether or not the determination of the loss ratio by 
the described lottery method and the proposed linear utility 
function instead of the assumed threshold loss function, 
however, are really improvements of the present decision 
component in MAIS (in terms of student performance on 



Some Concluding Remarks 
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posttosts, learning time, and amount of instruction) must be 
decided on the basis of experiments . Research projects in 
these areas have already been planned. 
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Table 1 

Twofold Table for Threshold Loss Function 



True level 



(true master) (true nonmaster) 



Decision 



Advance 



Retain 
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Figure Caption 

Figure 1 . An Example of a Linear Utility Function, 
(bg ^ bi, do * di) 
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